Expectation Maximization for Combined Phylogenetic and Hidden Markov Models

An expectation maximization (EM) algorithm is derived to estimate the parameters of a phylogenetic model, a probabilistic model of molecular evolution that considers the phylogeny, or evolutionary tree, by which a set of present-day organisms are related. The EM algorithm is then extended for use with a combined phylogenetic and hidden Markov model. An efficient method is also shown for computing gradients of the log likelihood of a phylogenetic model, which is similar in spirit to the EM algorithms, but has applications beyond EM—for example, in improving the efficiency of the quasi-Newton or conjugategradient methods that are commonly used to fit phylogenetic models. Finally, a small set of experiments are discussed in which the time to fit a phylogenetic model is compared among the EM algorithm, a standard quasi-Newton algorithm, and a modified quasi-Newton algorithm that computes gradients in the new, more efficient way. The performance of all algorithms is compared with that of PAML, a widely used software package for phylogenetic modeling. The modified quasi-Newton and EM algorithms both improve considerably on the standard quasi-Newton algorithm, and are competitive with PAML, despite that little care has been given to performance tuning. The EM algorithm seems particularly promising, despite somewhat slow convergence and a tendency occasionally to become “stuck” for several iterations at suboptimal parameter values. These problems may be addressed by the “entropic update” of Singer and Warmuth, which remains to be implemented and tested.